Hilo helps Quebecers optimize energy use by predicting demand at peak times with Azure Data Explorer

Quebec, Canada’s second-largest province, stays at the forefront of sustainability, generating 99 percent of energy from renewable sources. However, long, harsh winters and the ongoing transition to electricity in transport and heavy industry put a strain on its electrical grid, which called for new ways to produce and preserve energy.

To tackle these challenges, Hilo now offers affordable smart home solutions. “Our mission is to help the energy transition in Quebec and support the power grid in the province during peak demand,” says Matthieu Guyonnet-Duluc, Platform Product Manager at Hilo. “To achieve that, we leverage Internet of Things (IoT) devices like smart thermostats, water heaters, electric vehicle charging points, and others.”

Flattening the energy curve

“We have a few hours a day during the winter where utilities face high demand because of the heating. That then requires the use of major power sources like gas turbines to keep up,” Guyonnet-Duluc explains. “To avoid that, we balance energy demand during peak events using IoT devices—for example by decreasing the set point of the smart thermostat during the morning or evening to decrease energy consumption at homes and commercial buildings.”

In 2019, the company launched a pilot program covering 1,000 households. “We started building a virtual power plant to monitor and manage the energy consumption of every home at the minute level,” Guyonnet-Duluc continues. “Thanks to continuous insights from millions of telemetry points, we can predict short-term demand spikes, proactively managing power consumption to help Quebecers consume less energy and pay less.”

Ushering in data-driven sustainability

To collect, store, and process telemetry data, the pilot originally used a popular relational database solution. “Our developers come from a traditional enterprise environment, so it was a natural choice,” says Oisin Grehan, Platform Architect at Hilo. “But while it is a very capable solution for structured, systematic data, it was not designed to support the low latency and massive data volumes typical of telemetry projects. We are talking about more than two million messages ingested every minute at peak times,“ he says. High scalability cost was another issue. “Every time we wanted to scale, we had to pay for dedicated CPU and resources, which was adding up quickly,” adds Grehan.

Looking to bypass that, Hilo moved historical data to another database and only kept current information in the old one. “This also didn’t work for us. The more data we gathered, the more capacity we had to reserve, so the costs became enormous,” Grehan explains. Then, Azure Data Explorer (ADX) came into view. “We wanted a reliable, intuitive, and powerful platform that could generate dashboards and be used by everyone, from power users to business analysts,” Grehan says. “ADX has given us all of that.”

Big data dashboards on demand

“All our clients, domestic and commercial, have a smart hub installed, which interfaces with smart meters used to monitor power usage,” Grehan explains. “Data collected from millions of sensors is sent to the Azure IoT Hub and translated into a uniform format understood by our internal systems.” The translated data feeds into a digital twin, or a 1:1 virtual representation of the power grid that Hilo manages. To precisely gauge the energy demand and consumption trends, Hilo’s teams go to ADX dashboards, retrieving and visualizing information using Kusto, a query language suitable for fast-moving streams of big data.

“ADX does incredible work at providing the capability to build rich dashboards for very specific use cases. We’re looking at trends, the amount of energy spent over certain hours on a particular day of the week,” Grehan says. The dashboards offer a cross-sectional view of metrics from different devices. “What makes them very powerful is the integrated graphing, diagramming, and charting,” Grehan continues. “Normally, you'd have to get these functions from multiple off-the-shelf components. But ADX gives us all of them in one shot: you open the console, call in data, and everything gets done for you.”

Affordable clean energy at scale

In terms of business benefits, ADX offers an excellent cost-to-performance ratio for Hilo’s workloads. “We currently store about 8 TB of data. Maintaining that in a relational database is possible but we’d have to invest a lot of money to get the horsepower that ADX and Kusto provide with ease,” reveals Grehan. “By relying on ADX, we are paying 20 times less than before.”   

Since the platform has been designed with IoT and big data applications in mind, it also offers Hilo the reliability the company needs to deliver seamless services. “I don't think we've had a single outage in the two years since we've used it,” Grehan says. But its biggest advantage is being transparent and easy to scale. “ADX can easily scale to accommodate the traffic and the increasing number of customers without us having to add more servers or tweak database indexes.”

By flexibly adapting to the growing demand for clean energy, Hilo can more rapidly fulfill its mission to ensure Quebec’s clean energy transition through improved consumption and management of energy use during peak periods in winter. “Azure Data Explorer makes it very easy to ramp up our data and services. Without it, we couldn’t be helping thousands of Quebecers balance energy use at that scale," Grehan sums up.